Projects in Awesome Lists tagged with dag

https://github.com/apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

airflow apache apache-airflow automation dag data-engineering data-integration data-orchestrator data-pipelines data-science elt etl machine-learning mlops orchestration python scheduler workflow workflow-engine workflow-orchestration

Last synced: 16 Dec 2024

https://github.com/apache/incubator-airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

airflow apache apache-airflow automation dag data-engineering data-integration data-orchestrator data-pipelines data-science elt etl machine-learning mlops orchestration python scheduler workflow workflow-engine workflow-orchestration

Last synced: 23 Nov 2024

https://github.com/argoproj/argo-workflows

Workflow Engine for Kubernetes

airflow argo argo-workflows batch-processing cloud-native cncf dag data-engineering gitops hacktoberfest k8s knative kubernetes machine-learning mlops pipelines workflow workflow-engine

Last synced: 16 Dec 2024

https://github.com/argoproj/argo

Workflow Engine for Kubernetes

airflow argo argo-workflows batch-processing cloud-native cncf dag data-engineering gitops hacktoberfest k8s knative kubernetes machine-learning mlops pipelines workflow workflow-engine

Last synced: 22 Nov 2024

https://argoproj.github.io/argo-workflows/

Workflow Engine for Kubernetes

airflow argo argo-workflows batch-processing cloud-native cncf dag data-engineering gitops hacktoberfest k8s knative kubernetes machine-learning mlops pipelines workflow workflow-engine

Last synced: 29 Oct 2024

https://github.com/sogou/workflow

C++ Parallel Computing and Asynchronous Networking Framework

consul dag http kafka mysql redis sogou tasking

Last synced: 16 Dec 2024

https://github.com/marimo-team/marimo

A reactive notebook for Python — run reproducible experiments, execute as a script, deploy as an app, and version with git.

artificial-intelligence dag data-science data-visualization dataflow developer-tools machine-learning notebooks pipeline python reactive web-app

Last synced: 16 Dec 2024

https://github.com/antvis/X6

🚀 JavaScript diagramming library that uses SVG and HTML for rendering.

antv dag diagram diagramming erd erdiagram flowchart graph graph-editor javascript javascript-diagramming-library svg typescript uml-diagram

Last synced: 31 Oct 2024

https://github.com/antvis/x6

🚀 JavaScript diagramming library that uses SVG and HTML for rendering.

antv dag diagram diagramming erd erdiagram flowchart graph graph-editor javascript javascript-diagramming-library svg typescript uml-diagram

Last synced: 16 Dec 2024

https://github.com/alibaba/butterfly

🦋Butterfly，A JavaScript/React/Vue2 Diagramming library which concentrate on flow layout field. (基于JavaScript/React/Vue2的流程图组件)

bpmn butterfly connectivity dag diagram diagramming edge flow flow-dag flow-layout flowchart mindmap react svg tree treemap-diagram vue

Last synced: 16 Dec 2024

https://github.com/hatchet-dev/hatchet

A distributed, fault-tolerant task queue

concurrency dag distributed distributed-systems durable-execution event-driven fastapi golang nodejs python queue typescript workflow-engine

Last synced: 17 Dec 2024

https://github.com/orchest/orchest

Build data pipelines, the easy way 🛠️

airflow cloud dag data-pipelines data-science deployment docker etl etl-pipeline ide jupyter jupyterlab kubernetes machine-learning notebooks orchest pipelines python self-hosted

Last synced: 18 Dec 2024

https://github.com/dagworks-inc/hamilton

Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

dag data-analysis data-engineering data-science dataframe etl etl-framework etl-pipeline feature-engineering hacktoberfest lineage llmops machine-learning mlops orchestration pandas python rag software-engineering

Last synced: 17 Dec 2024

https://github.com/DAGWorks-Inc/hamilton

Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

dag data-analysis data-engineering data-science dataframe etl etl-framework etl-pipeline feature-engineering hacktoberfest lineage llmops machine-learning mlops orchestration pandas python rag software-engineering

Last synced: 29 Oct 2024

https://github.com/chunelfeng/cgraph

【A common used C++ DAG framework】一个通用的、无三方依赖的、跨平台的、收录于awesome-cpp的、基于流图的并行计算框架。欢迎star & fork & 交流

ai dag flow graph pipeline taskflow threadpool workflow

Last synced: 19 Dec 2024

https://github.com/teamclairvoyant/airflow-maintenance-dags

A series of DAGs/Workflows to help maintain the operation of Airflow

airflow airflow-maintenance-dags apache-airflow cleanup dag maintenance workflow

Last synced: 19 Dec 2024

https://github.com/dtstack/taier

Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display

azkaban chunjun cronjob-scheduler dag data-schedule distributed-schedule-system flink hadoop hive job-scheduler scheduler spark task-schedule workflow-scheduling-system

Last synced: 18 Dec 2024

https://github.com/DTStack/Taier

Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display

azkaban chunjun cronjob-scheduler dag data-schedule distributed-schedule-system flink hadoop hive job-scheduler scheduler spark task-schedule workflow-scheduling-system

Last synced: 30 Oct 2024

https://github.com/ChunelFeng/CGraph

【A simple C++ DAG framework】一个简单好用的、无三方依赖的、跨平台的、收录于awesome-cpp的、基于流图的并行计算框架。欢迎star & fork

ai awesome-cpp bilibili concurrency cpp dag framework graph hpc machine-vision taskflow threadpool workflow

Last synced: 27 Oct 2024

https://github.com/bellaj/blockchain

Compilation of useful documents and scientific papers about Blockchain & cryptocurrencies.

aeternity bitcoin bitcoin-ethereum blockchain corda cryptocurrencies cryptography dag ethereum fractal hyperledger mchain quorum whitepapers

Last synced: 15 Dec 2024

https://github.com/Qiskit/rustworkx

A high performance Python graph library implemented in Rust.

dag graph graph-theory python rust

Last synced: 30 Oct 2024

https://github.com/caicloud/cyclone

Powerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resources. https://cyclone.dev

ai-pipeline ci-cd continuous-delivery continuous-integration dag kubernetes workflow

Last synced: 21 Dec 2024

https://github.com/swimlane/ngx-graph

Graph visualization library for angular

angular angular2 chart dag data-viz directed-graph svg workflow

Last synced: 17 Dec 2024

https://github.com/paddlepaddle/serving

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

dag deep-learning docker gpu micro-service microservice-toolkit online-service paddle paddle-serving pipeline prediction predictor python rpc-service serving

Last synced: 20 Dec 2024

https://github.com/PaddlePaddle/Serving

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

dag deep-learning docker gpu micro-service microservice-toolkit online-service paddle paddle-serving pipeline prediction predictor python rpc-service serving

Last synced: 27 Oct 2024

https://github.com/stitchfix/hamilton

A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton

dag data-engineering data-platform data-science dataframe etl etl-framework etl-pipeline feature-engineering featurization hamilton hamiltonian machine-learning numpy pandas python software-engineering stitch-fix

Last synced: 26 Sep 2024

https://github.com/spacemeshos/go-spacemesh

Go Implementation of the Spacemesh protocol full node. 💾⏰💪

blockchain blockmesh consensus cryptocurrencies dag decentralization go golang p2p proof-of-space proof-of-space-time proof-of-work spacemesh

Last synced: 17 Dec 2024

https://github.com/deepcs233/jieba_fast

Use C Api and Swig to Speed up jieba 高效的中文分词库

dag jieba python swig viterbi-hmm

Last synced: 15 Dec 2024

https://github.com/antvis/XFlow

React component for building interactive diagrams.

dag flowchart react x6

Last synced: 25 Nov 2024

https://github.com/antvis/xflow

React component for building interactive diagrams.

dag flowchart react x6

Last synced: 20 Dec 2024

https://github.com/MixinNetwork/mixin

🚀 The Mixin TEE-BFT-DAG network reference implementation.

bft bitcoin blockchain dag lightning-network mixin mixinnetwork

Last synced: 15 Nov 2024

https://github.com/erdogant/bnlearn

Python package for Causal Discovery by learning the graphical structure of Bayesian networks. Structure Learning, Parameter Learning, Inferences, Sampling methods.

bayesian-inference causal causal-models causal-networks causality-analysis dag directed-acyclic-graph machine-learning parameter-learning python sampling-methods structure-learning

Last synced: 19 Dec 2024

https://github.com/malcolmbarrett/ggdag

:arrow_lower_left: :arrow_lower_right: An R package for working with causal directed acyclic graphs (DAGs)

causal-inference dag ggplot-extension r rstats

Last synced: 07 Oct 2024

https://github.com/r-causal/ggdag

:arrow_lower_left: :arrow_lower_right: An R package for working with causal directed acyclic graphs (DAGs)

causal-inference dag ggplot-extension r rstats

Last synced: 25 Oct 2024

https://github.com/flowerfine/scaleph

Open data platform based on Kubernetes. Scaleph supports SeaTunnel、Flink and Doris backended by SeaTunnel on Flink engine、Flink Kubernetes Operator and Doris operator.

dag data-platform dataops doris doris-manager doris-operator flink flink-kubernetes flink-kubernetes-operator flink-sql flink-sql-gateway seatunnel

Last synced: 05 Nov 2024

https://github.com/xdagger/xdag

XDAG (Dagger Coin) Official Main Repository. XDAG is a novel DAG based cryptocurrency.

bitcoin blockchain blockchain-technology cryptocurrency dag dagger xdag xdagger

Last synced: 15 Dec 2024

https://github.com/XDagger/xdag

XDAG (Dagger Coin) Official Main Repository. XDAG is a novel DAG based cryptocurrency.

bitcoin blockchain blockchain-technology cryptocurrency dag dagger xdag xdagger

Last synced: 07 Nov 2024

https://github.com/weibocom/rill-flow

Rill Flow is a high-performance, scalable workflow orchestration engine for distributed workloads and LLMs

agent ai backend-as-a-service cloud-native dag job-scheduler llm orchestration task-scheduler workflow

Last synced: 15 Dec 2024

https://github.com/vitelabs/go-vite

Official Go implementation of the Vite protocol

blockchain cryptocurrency dag golang ledger smart-contracts vite

Last synced: 15 Dec 2024

https://github.com/csunny/py-bitcoin

比特币的python简易实现，主要介绍比特币底层技术。如Base58编码、椭圆加密算法、MerkleTree、P2P对等网络、RPC通信、UTXO、虚拟机、DHT、DAG、链上数据的持久化存储等。

bitcoin dag dpos merkletree p2p-network pos pow python3 utxo

Last synced: 17 Dec 2024

https://github.com/intel/dffml

The easiest way to use Machine Learning. Mix and match underlying ML libraries and data set sources. Generate new datasets or modify existing ones with ease.

ai-inference ai-machine-learning ai-training analytics asyncio dag data-flow dataflows datasets dffml event-based flow-based-programming frameworks hyperautomation libraries machine-learning models pipelines python swrepo

Last synced: 15 Nov 2024

https://github.com/snowplow-archive/factotum

A system to programmatically run data pipelines

cron dag job job-scheduler rust

Last synced: 06 Dec 2024

https://github.com/pipefunc/pipefunc

Lightweight function pipeline (DAG) creation in pure Python for scientific workflows 🕸️🧪

dag hpc parallel-computing pipeline-framework pipelines reproducible-research slurm workflow-engine

Last synced: 21 Dec 2024

https://github.com/Barski-lab/cwl-airflow

Python package to extend Airflow functionality with CWL1.1 support

common-workflow-language commonwl containers cwl cwl-airflow cwl-descriptor dag workflow

Last synced: 25 Nov 2024

https://github.com/josephg/statecraft

Manage state with finesse

dag database event-sourcing realtime-collaboration realtime-database

Last synced: 19 Dec 2024

https://github.com/coti-io/coti-node

COTI is the first DAG based chain protocol that's optimized for Enterprises and Stable-coins.

blockchain coti coti-node cryptocurrency dag java p2p payments

Last synced: 17 Nov 2024

https://github.com/nicor88/aws-ecs-airflow

Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks

airflow airflow-ecs aws dag docker ecr ecs etl fargate terraform

Last synced: 21 Dec 2024

https://github.com/ngneat/dag

🐠 An Angular service for managing directed acyclic graphs

angular dag directed-acyclic-graph graph

Last synced: 19 Nov 2024

https://github.com/ucbrise/flor

🌻 FlorFlow: Flor, now with Dataflow

airflow build dag deep-learning flor hindsight logger logging machine-learning ml pytorch tensorboard vldb

Last synced: 15 Dec 2024

https://github.com/MyButifullyLife/vue-x6-flow

vue版流程图简易编辑器、简单易上手

antv antv-x6 dag flow vue vue-flow-edit

Last synced: 28 Oct 2024

https://github.com/allegro/bigflow

A Python framework for data processing on GCP.

airflow-dag beam bigquery composer dag dataflow dataproc gcp python python-framework workflows

Last synced: 21 Dec 2024

https://github.com/xdagger/xdagj

XDAGJ is an implementation of XDAG in Java. https://xdag.io

asic-resistant blockchain consensus-protocol cryptocurrency dag distributed-ledger-technology java libp2p metaverse mining-pool mnemonic p2p-network randomx rocksdb-java rpc-api snapshot sponsorship wallets

Last synced: 21 Dec 2024

https://github.com/ipfs/js-dag-service

Library for storing and replicating hash-linked data over the IPFS network.

dag decentralized distributed ipfs ipld p2p peer-to-peer

Last synced: 21 Dec 2024

https://github.com/feluelle/airflint

Enforce Best Practices for all your Airflow DAGs. ⭐

airflow best-practices dag formatter linter

Last synced: 21 Dec 2024

https://github.com/mindee/tawazi

A DAG Scheduler library written in pure python

dag graph parallel python

Last synced: 21 Dec 2024

https://github.com/lchrennew/dragonfly-dag

完全支持Vue3和Vitejs的DAG流程图组件

dag diagram flowchart vitejs vue3

Last synced: 07 Nov 2024

https://github.com/vesoft-inc/nebulagraph-veditor

A Highly customizable JavaScript Diagramming Lib with SVG, HTML, Canvas support.

canvas dag diagram erd erdiagram flowchart graph javascript nebulagraph svg

Last synced: 07 Nov 2024

https://github.com/domenp/aircal

Visualize Airflow's schedule by exporting future DAG runs as events to Google Calendar.

airflow calendar dag google-calendar schedule

Last synced: 22 Nov 2024

https://github.com/ipfs-shipyard/ipfs-geoip

geoip lookup over DAG-CBOR dataset loaded from IPFS

b-tree dag dag-cbor geoip ipfs

Last synced: 12 Nov 2024

https://github.com/zflow-dev/zflow

Flow Based Low/No Code Workflow and Robotic Process Automation Library

dag fbp flowbased low-code no-code rust visual-programming

Last synced: 09 Dec 2024

https://github.com/elpaso/django-dag

Basic portable Directed Acyclic Graph application for Django

dag django

Last synced: 27 Oct 2024

https://github.com/fristonio/xene

A distributed workflow runner focusing on performance and simplicity.

airflow cloud dag distributed-systems docker golang kubernetes workflow workflow-engine

Last synced: 07 Nov 2024

https://github.com/zhenv5/breaking_cycles_in_noisy_hierarchies

breaking cycles in noisy hierarchies

bfs break-cycles cycle-edges dag dfs directed-acyclic-graph graph-hierarchy minimum-feedback-arc-set socialagony trueskill

Last synced: 28 Nov 2024

https://github.com/grame-cncm/digraph

Very simple C++ directed graph library

cycle dag digraph graph graph-algorithms tarjan

Last synced: 10 Nov 2024

https://github.com/capillariesio/capillaries

Distributed batch data processing framework

batch-processing cassandra dag distributed-computing distributed-systems go golang rabbitmq relational-algebra workflow-engine workflows

Last synced: 20 Dec 2024

https://github.com/zhongantech/annchain.og

Annchain.OG is a DAG based decentralized ledger and Dapp platform

blockchain dag dapp ledger

Last synced: 11 Nov 2024

https://github.com/realrate/causing

Causing: CAUsal INterpretation using Graphs

automatic-differentiation causal-networks causality-analysis dag derivatives do-calculus effects-modeling gnn graph-theory graphs latent-variables mediation-analysis neural-networks python pytorch simultaneous-equation structural-analysis structural-equation-modeling

Last synced: 21 Dec 2024

https://github.com/sutd-visual-computing-group/dag-gans

Data Augmentation optimized for GAN

biggan cyclegan dag data-augmentation gan pytorch tensorflow

Last synced: 15 Nov 2024

https://github.com/salesforce/apollo

An experimental multi-tenant distributed system platform

consensus dag distributed-database gossip-protocol secure-membership

Last synced: 08 Nov 2024

https://github.com/extensible-ai/dagent

Build AI Agents with Your Existing Python Code!

agent agents ai ai-agents dag framework llama llm ollama openai python

Last synced: 27 Oct 2024

https://github.com/aleximb/vue-dag

🏗 Data-driven directed acyclic graph (DAG) builder for Vue.js

dag directed-acyclic-graph typescript vue

Last synced: 17 Dec 2024

https://github.com/loehnertz/tanglestash

IOTA meets BitTorrent: An algorithm to persist any file onto the tangle of IOTA

dag distributed-storage filesharing iota nodejs storage tangle torrent

Last synced: 20 Oct 2024

https://github.com/loehnertz/Tanglestash

IOTA meets BitTorrent: An algorithm to persist any file onto the tangle of IOTA

dag distributed-storage filesharing iota nodejs storage tangle torrent

Last synced: 03 Nov 2024

https://github.com/ddddddo/gdag

Easily manage 🕸DAG🕷 with Go. DAG is an acronym for Directed Acyclic Graph. Output is in PlantUML or Mermaid format. Useful for progressing tasks.

dag data-structures mermaid plantuml workflow

Last synced: 18 Nov 2024

https://github.com/omenapps/django-postgresql-dag

Directed Acyclic Graphs with a variety of methods for both Nodes and Edges, and multiple exports (NetworkX, Pandas, etc). This project is the foundation for a commercial product, so expect regular improvements. PR's and other contributions are welcomed.

common-table-expression cte dag directed-acyclic-graph directed-graph django graph graph-algorithms postgresql

Last synced: 01 Nov 2024

https://github.com/autotraderuk/oliver-twist

dbt DAG Auditor

dag dag-auditing dbt

Last synced: 05 Nov 2024

https://github.com/hyperbadger/nomad-pipeline

Run pipeline-style workloads in Nomad

dag hashicorp nomad pipeline

Last synced: 15 Dec 2024

https://github.com/storacha/pail

🪣 DAG based key value store.

bucket crdt dag ipfs ipld kv kv-store merkle-crdt merkle-dag shard

Last synced: 08 Nov 2024

https://github.com/parthsareen/dagent

Create AI Agents Quickly

agent agents ai ai-agents dag framework llama llm ollama openai python

Last synced: 09 Oct 2024

https://github.com/hadron-project/hadron

The Kubernetes native and CloudEvents native distributed event streaming, event orchestration & messaging platform

cloudevents dag event-driven events kubernetes messaging orchestration pipelines pubsub rpc rust streams workflow-orchestration workflows

Last synced: 11 Nov 2024

https://github.com/ww-tech/primrose

Primrose modeling framework for simple production models

dag data-science datascience deployment machine-learning primrose python workflows

Last synced: 18 Nov 2024

https://github.com/telkins/laravel-dag-manager

A SQL-based Directed Acyclic Graph (DAG) solution for Laravel.

dag directed-acyclic-graph hacktoberfest laravel sql

Last synced: 16 Dec 2024

https://github.com/sambrightman/asciidag

Draw DAGs (directed acyclic graphs) as ASCII art, à la Git

ascii ascii-art dag directed-acyclic-graph gitgraph python python2 python3 tree

Last synced: 13 Oct 2024

https://github.com/bonustrack/obyte.js

A pure and powerful JavaScript Obyte library.

api byteball dag javascript obyte oscript

Last synced: 09 Nov 2024

https://github.com/moaxcp/graph-dsl

A groovy dsl for creating and traversing graphs.

dag directed-graph graph groovy

Last synced: 08 Nov 2024

https://github.com/glebec/batching-toposort

Efficiently sort interdependent tasks into a sequence of concurrently-executable batches

algorithm concurrency dag digraph directed-acyclic-graph graph sort toposort

Last synced: 28 Oct 2024

https://github.com/Setheum-Labs/Setheum

Setheum: The Unified Web3 Base Layer - Globally Solving Web3’s Fragmentation Problems. Ready for hacking 🚀

blockchain blockchain-dag contracts cryptocurrency dag defi erc20 evm light-speed setheum setheum-network smart smart-contracts solidity substrate

Last synced: 01 Nov 2024

https://github.com/wittline/pydag

Scheduling Big Data Workloads and Data Pipelines in the Cloud with pyDag

big-data bigquery cloud dag data-engineering data-pipeline dataengineering dataproc dataproc-cluster directed-acyclic-graph google-cloud google-cloud-platform parallel-processing task-scheduler task-scheduling workflow-engine

Last synced: 14 Oct 2024

https://github.com/tableflip/ipld-explorer-cli

🔎 Explore the IPLD directed acyclic graph with your keyboard

acyclic cli dag directed explore forest graph hash ipld link merkle node repl tree

Last synced: 05 Nov 2024

https://github.com/XDagger/DaggerGpuMiner

Standalone GPU/CPU miner for Dagger coin

dag dagger daggerminer gpu gpuminer miner xdag

Last synced: 07 Nov 2024

https://github.com/atheler/klang

Block based synthesis and music library for Python

adsr dag envelope frequency-modulation micro-rhythm midi music music-composition music-generation music-library music-theory oscillator python python3 rhythm synth synthesizer temperament tuning waveform

Last synced: 13 Oct 2024

https://github.com/astronomer/airflow-provider-fivetran-async

A new Airflow Provider for Fivetran, maintained by Astronomer and Fivetran

airflow airflow-operator airflow-provider apache-airflow dag data-orchestration etl python workflow

Last synced: 19 Nov 2024

https://github.com/zifeo/whiz

Modern DAG/tasks runner for multi-platform monorepos with live reloading, env management, pipes, and more in a tabbed view.

build-tool dag logger monorepo platform task-runner watcher

Last synced: 25 Nov 2024

https://github.com/joscha/buildkite-graph

A graph-based generator for Buildkite pipelines

buildkite buildkite-pipelines dag dependencies dynamic-pipelines graph serialization yaml

Last synced: 02 Nov 2024

https://github.com/yeaseen/c-compiler

:hamburger: A subset of C Compiler[Lexical Analyzer, Syntax Analyzer, Semantic Analyzer & Intermediate Code Generator] , DAG & TAC implemented in C++ using Flex and Yacc-Bison as an assignment of sessional course CSE310 in undergradute studies in CSE, BUET

assembly-8086 bison c-compiler compiler-construction compiler-design cse-buet dag intermediate-code-generation lexical-analyzer parser parser-generator semantic-analyzer symbol-table tac yacc