Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/areller/denogent
A TypeScript build system
build build-tool ci deno pipelines typescript
Last synced: 03 Jul 2024
https://github.com/logrange/logrange
High performance data aggregating storage
database go golang log-aggregation logging pipelines stream-processing streams
Last synced: 02 Jul 2024
https://github.com/typestreamio/typestream
Open Source streaming platform. Write and run typed data pipelines with a minimal, familiar syntax.
Last synced: 01 Jul 2024
https://github.com/Hydrospheredata/hydro-serving
MLOps Platform
machine-learning models pipelines realtime scikit-learn scoring serverless serving spark tensorflow
Last synced: 29 Jun 2024
https://github.com/WaylonWalker/find-kedro
kedro plugin to automatically construct pipelines using pytest style pattern matching
kedro kedro-hook kedro-plugin pipelines python
Last synced: 26 Jun 2024
https://github.com/hahwul/gee
🏵 Gee is tool of stdin to each files and stdout. It is similar to the tee command, but there are more functions for convenience. In addition, it was written as go
bugbounty-tool command-line-tool golang pipelines security-tools tee utility
Last synced: 25 Jun 2024
https://github.com/AgnostiqHQ/covalent
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
covalent data-pipeline data-science deep-learning hacktoberfest hpc hpc-applications machine-learning machinelearning machinelearning-python orchestration parallelization pipelines python quantum quantum-computing quantum-machine-learning workflow workflow-automation workflow-management
Last synced: 23 Jun 2024
https://argoproj.github.io/argo-workflows/
Workflow Engine for Kubernetes
airflow argo argo-workflows batch-processing cloud-native cncf dag data-engineering gitops hacktoberfest k8s knative kubernetes machine-learning mlops pipelines workflow workflow-engine
Last synced: 22 Jun 2024
https://github.com/benc-uk/terraform-mgmt-bootstrap
Bootstrap core Azure state & resources using Terraform for use with Azure DevOps
azure azure-cli azure-devops devops infrastructure-as-code pipelines terraform
Last synced: 21 Jun 2024
https://github.com/PennLINC/qsiprep
Preprocessing and reconstruction of diffusion MRI
connectomics diffusion-mri pipelines python tractography
Last synced: 20 Jun 2024
https://github.com/connectomicslab/connectomemapper3
Connectome Mapper 3 is a BIDS App that implements full anatomical, diffusion, resting/state functional MRI, and recently EEG processing pipelines, from raw T1 / DWI / BOLD , and preprocessed EEG data to multi-resolution brain parcellation with corresponding connection matrices.
bids bids-apps bidsapp brain chuv connectome dmri docker eeg fmri parcellation pipelines python singularity
Last synced: 20 Jun 2024
https://github.com/dataplane-app/dataplane
Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.
airflow data data-analysis data-engineering data-integration data-pipelines data-science dataplane datawarehouse etl finance golang kubernetes pipelines robotics-process-automation rpa scheduler workflow workflow-automation workflows
Last synced: 17 Jun 2024
https://github.com/redhat-scholars/tekton-tutorial
Get started with Tekton Pipelines :smiley_cat: for https://dn.dev/master. Learn from basics to advanced
cd ci cli demos kubernetes pipelines redhat-developers tekton-pipelines tekton-triggers tektoncd triggers tutorial
Last synced: 13 Jun 2024
https://github.com/molu8bits/jenkins-security-pipelines
Jenkins security pipelines
declarative devsecops jenkins pipelines security zap
Last synced: 13 Jun 2024
https://github.com/mikhailknyazev/kube-course
Main resources for Udemy course "Configuring Kubernetes for Reliability with LitmusChaos"
chaos-engineering eks helm kubernetes litmuschaos pipelines reliability terraform
Last synced: 12 Jun 2024
https://github.com/stacksimplify/azure-aks-kubernetes-masterclass
Azure AKS Kubernetes Masterclass
aks azure-container-registry azure-devops kubernetes pipelines
Last synced: 12 Jun 2024
https://github.com/DamZiobro/gstreamerCheatsheet
GStreamer pipelines and CLI commands for different GStreamer based features (process MPEG2-TS files, get video from DVB, deinterlace video, capture RTSP stream etc.)
capture-rtsp-stream capture-screen cheatsheet dvb gst gstreamer gstreamer-pipeline hls mpeg mpegts pipelines rtsp video youtube
Last synced: 11 Jun 2024
https://github.com/live-keys/livekeys
Livekeys - Automation Platform with support for live scripting.
image-processing live-coding pipelines video-processing
Last synced: 11 Jun 2024
https://github.com/HoussemDellai/build-agents-in-containers
Demo for running Azure Pipelines inside Docker Containers in a host machine or in Kubernetes/AKS with KEDA to support horizontal scalability.
azure azure-devops devops keda pipelines scalability
Last synced: 10 Jun 2024
https://github.com/flemay/3musketeers
Test, build, and deploy your apps from anywhere, the same way!
3musketeers ci cicd cloudflare-pages devcontainer docker docker-compose documentation github-actions make makefile methodology microservice monolithic pipelines repeatability typescript
Last synced: 09 Jun 2024
https://github.com/Avaiga/taipy
Turns Data and AI algorithms into production-ready web applications in no time.
automation data-engineering data-integration data-ops data-visualization datascience developer-tools hacktoberfest hacktoberfest2023 job-scheduler mlops orchestration pipeline pipelines python scenario scenario-analysis taipy-core taipy-gui workflow
Last synced: 08 Jun 2024
https://github.com/mrpaulandrewltd/Microsoft-Data-Integration-Pipeline-Training
Training workshop content on Azure Data Factory and Azure Synapse Analytics Data Integration Pipelines
azure data data-factory integration pipelines procfwk synapse-analytics
Last synced: 04 Jun 2024
https://github.com/mrpaulandrew/procfwk
A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.
adf adfprocfwk azure azure-functions azure-sql-database data-engineering data-factory framework metadata pipelines processing procfwk
Last synced: 04 Jun 2024
https://github.com/mlr-org/mlr3pipelines
Dataflow Programming for Machine Learning in R
bagging data-science dataflow-programming ensemble-learning machine-learning mlr3 pipelines preprocessing r r-package stacking
Last synced: 04 Jun 2024
https://github.com/Azure/azure-devops-cli-extension
Azure DevOps Extension for Azure CLI
agile artifacts azure azure-cli azure-cli-extension azure-devops azure-devops-extension boards cicd command-line command-line-tool devops git pipelines repos
Last synced: 03 Jun 2024
https://github.com/pytorch/torchx
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
airflow aws-batch components deep-learning distributed-training kubernetes machine-learning pipelines python pytorch ray slurm
Last synced: 01 Jun 2024
https://github.com/pypyr/pypyr
pypyr task-runner cli & api for automation pipelines. Automate anything by combining commands, different scripts in different languages & applications into one pipeline process.
automation cd ci ci-cd continuous-deployment continuous-integration devops pipeline pipeline-processor pipeline-runner pipelines pipelines-yaml python script script-loader task-manager task-runner taskrunner tool
Last synced: 31 May 2024
https://github.com/beetlex-io/BeetleX.Light
Based on pipelines high performance dotnet core socket tcp/udp communication components
asyncsocket non-blocking pipelines socket socket-client socket-communication socket-programming socket-server socketasynceventargs tcp tcp-client tcp-server udp udp-client udp-server udp-socket
Last synced: 31 May 2024
https://github.com/meltano/meltano
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
connectors data data-engineering data-pipelines dataops dataops-platform elt extract-data integration loaders meltano meltano-sdk open-source opensource pipelines singer tap taps target targets
Last synced: 31 May 2024
https://github.com/nodamu/sign-language-recogntion
Cross platform sign language translation with mediapipe
computer-vision hand-gesture-recognition mediapipe pipelines
Last synced: 29 May 2024
https://github.com/Jean-njoroge/Breast-cancer-risk-prediction
Classification of Breast Cancer diagnosis Using Support Vector Machines
breast-cancer-prediction breast-cancer-tumor breastcancer-classification classification data-analysis dataprocessing exploratory-data-analysis notebook pipelines prediction-model python supervised-learning svm
Last synced: 29 May 2024
https://github.com/sharpliner/sharpliner
Use C# instead of YAML to define your Azure DevOps pipelines
azure-devops azure-devops-pipelines csharp dotnet pipelines yaml
Last synced: 28 May 2024
https://github.com/temptemp3/ba.sh
public bash scripts used in around the web
array arrays bash bash-script bash-scripting export functions map pipeline pipelines pipelining pipes pipes-sh shell shell-script shell-scripting shell-scripts shellcode tests xargs
Last synced: 26 May 2024
https://github.com/ManchesterBioinference/Scasat
Scasat is a single cell ATAC-seq preprocessing and analysis pipeline
atac-seq pipelines single-cell
Last synced: 24 May 2024
https://github.com/onepanelio/onepanel
The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.
ai aiops annotation computer-vision deeplearning etl hyperparameter-tuning inference jupyterlab labeling machinelearning mlops pipelines pytorch tensorboard tensorflow training workflows
Last synced: 19 May 2024
https://github.com/shini4i/argo-watcher
A service designed to enhance the monitoring and visibility of deployments managed through ArgoCD Image Updater. Additionally, it supports making commits directly to the GitOps repository, providing an alternative to using the image updater.
argocd automation ci-cd devops gitops kubernetes pipelines
Last synced: 19 May 2024
https://github.com/DevOpsEnabled/XConfigMaster
devops-pipeline-language flexible language pipelines
Last synced: 17 May 2024
https://github.com/4dn-dcic/tibanna
Tibanna helps you run your genomic pipelines on Amazon cloud (AWS). It is used by the 4DN DCIC (4D Nucleome Data Coordination and Integration Center) to process data. Tibanna supports CWL/WDL (w/ docker), Snakemake (w/ conda) and custom Docker/shell command.
aws aws-lambda aws-step-function bioinformatics conda cwl cwl-workflow docker pipelines snakemake snakemake-workflow wdl wdl-workflow
Last synced: 17 May 2024
https://github.com/orchest/orchest
Build data pipelines, the easy way 🛠️
airflow cloud dag data-pipelines data-science deployment docker etl etl-pipeline ide jupyter jupyterlab kubernetes machine-learning notebooks orchest pipelines python self-hosted
Last synced: 16 May 2024
https://github.com/timoschwarzer/gitlab-monitor
A web-based monitor dashboard for GitLab CI
dashboard gitlab gitlabci hacktoberfest monitor pipelines
Last synced: 14 May 2024
https://github.com/patterns-app/patterns-devkit
Data pipelines from re-usable components
data-analysis data-engineering data-pipeline data-pipelines data-science etl etl-framework etl-pipeline etl-pipelines functional-reactive-programming immutability pipelines sql
Last synced: 13 May 2024
https://github.com/argoproj/argo-events
Event-driven Automation Framework for Kubernetes
argo automation-framework cloud-native cloudevents event-driven event-source eventing-framework kubernetes pipelines triggers workflow-automation workflows
Last synced: 13 May 2024
https://github.com/cefriel/chimera
Composable Semantic Transformation Pipelines
apache-camel etl-framework mediator pipelines rdf semantic-web
Last synced: 13 May 2024
https://github.com/Cloud-Pipelines/pipeline-editor
Cloud Pipelines Editor is a web app that allows the users to build and run Machine Learning pipelines without having to set up development environment.
google-cloud google-cloud-platform google-cloud-vertex google-cloud-vertex-ai kubeflow-pipelines machine-learning pipeline pipelines vertex-ai
Last synced: 12 May 2024
https://github.com/TrainingByPackt/Command-Line-Fundamentals
Learn to use the Unix command-line tools and Bash shell scripting
bash-shell command-line pipelines redirection shell-scripting
Last synced: 11 May 2024
https://github.com/JuliaAI/MLJ.jl
A Julia machine learning framework
classification clustering data-science ensemble-learning julia machine-learning pipeline pipelines predictive-modeling regression stacking statistics tuning tuning-parameters
Last synced: 11 May 2024
https://github.com/salesforce/TransmogrifAI
TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning
ai automated-machine-learning automl dsl einstein estimators feature-engineering features machine-learning ml pipelines salesforce scala spark sparkml structured-data transformations transformers transmogrification transmogrify
Last synced: 09 May 2024
https://github.com/concourse/concourse
Concourse is a container-based continuous thing-doer written in Go.
ci ci-cd concourse continuous-delivery continuous-integration elm go hacktoberfest pipelines
Last synced: 08 May 2024
https://github.com/tektoncd/triggers
Event triggering with Tekton!
hacktoberfest pipelines tekton triggers
Last synced: 08 May 2024
https://github.com/hebes-io/eensight
The measurement and verification methodology of the H2020 project SENSEI
building-energy energy-data energy-efficiency kedro pipelines
Last synced: 08 May 2024
https://github.com/elyra-ai/elyra
Elyra extends JupyterLab with an AI centric approach.
ai airflow anaconda apache-airflow binder docker elyra hacktoberfest jupyterlab jupyterlab-extension jupyterlab-extensions jupyterlab-notebooks kubeflow kubeflow-pipelines machine-learning notebook-jupyter notebooks pipelines pypi python
Last synced: 06 May 2024
https://github.com/ploomber/ploomber
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
data-engineering data-science jupyter jupyter-notebooks machine-learning mlops notebooks papermill pipelines pycharm vscode workflow
Last synced: 06 May 2024
https://github.com/ivanpaulovich/FluentMediator
:twisted_rightwards_arrows: FluentMediator is an unobtrusive library that allows developers to build custom pipelines for Commands, Queries and Events.
chain-methods chain-of-responsibility craftmanship csharp ddd-cqrs dotnet-core event-driven event-handlers event-sourcing fluent-design fluent-interface mediator-pattern message-bus pipeline-framework pipelines tdd
Last synced: 05 May 2024
https://github.com/polyaxon/polyaxon
MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle
artificial-intelligence caffe data-science deep-learning hyperparameter-optimization jupyter jupyterlab k8s keras kubernetes machine-learning ml mlops mxnet notebook pipelines pytorch reinforcement-learning tensorflow workflow
Last synced: 05 May 2024
https://github.com/ctuning/ck-tensorflow
Collective Knowledge components for TensorFlow (code, data sets, models, packages, workflows):
apis automation collective-knowledge knowledge-management metadata package-management pipelines workflow-automation
Last synced: 02 May 2024
https://github.com/smooks/smooks
Extensible data integration Java framework for building XML and non-XML fragment-based applications
analytics big-data enterprise-integration etl event-driven java pipelines sax smooks stream-processing xml
Last synced: 02 May 2024
https://github.com/zorbash/opus
A framework for pluggable business logic components
dsl elixir pipelines railway-oriented-programming
Last synced: 01 May 2024
https://github.com/gobeam/mongo-go-pagination
Golang Mongodb Pagination for official mongodb/mongo-go-driver package which supports both normal queries and Aggregation pipelines with all information like Total records, Page, Per Page, Previous, Next, Total Page and query results.
aggregate aggregation database driver go go-mongo golang match mongo mongo-go-driver mongo-go-pagination mongo-pagination mongodb mongodb-driver pagination pagination-library paginator perpage pipelines totalpage
Last synced: 29 Apr 2024
https://github.com/slok/brigade-exporter
Exporter for brigade metrics
brigade ci ci-cd kubernetes metrics observability pipelines prometheus
Last synced: 29 Apr 2024
https://github.com/argoproj/argo-workflows
Workflow Engine for Kubernetes
airflow argo argo-workflows batch-processing cloud-native cncf dag data-engineering gitops hacktoberfest k8s knative kubernetes machine-learning mlops pipelines workflow workflow-engine
Last synced: 26 Apr 2024
https://github.com/thomhurst/ModularPipelines
Write your pipelines in C# !
build build-system build-tool build-tools csharp dotnet dotnet-core dotnetcore pipeline pipeline-deploy pipeline-framework pipelines
Last synced: 25 Apr 2024
https://github.com/ERNI-Academy/assets-azuredevops-pipelines
Set of jobs and templates of ymal files to be used in Azure DevOps pipelines
azure azure-devops ci-cd cicd continuous-delivery continuous-deployment continuous-integration devops pipeline pipelines
Last synced: 23 Apr 2024
https://github.com/eora-ai/inferoxy
Service for quick deploying and using dockerized Computer Vision models
computer-vision machine-learning mlops pipelines python
Last synced: 23 Apr 2024
https://github.com/dmesquita/dvc_pipelines_and_experiments_tutorial
Building a maintainable Machine Learning pipeline using DVC
dvc dvc-for-experiment-management experiments machine-learning pipelines project-management reproducibility
Last synced: 22 Apr 2024
https://github.com/mage-ai/mage-ai
🧙 Build, run, and manage data pipelines for integrating and transforming data.
artificial-intelligence data data-engineering data-integration data-pipelines data-science dbt elt etl machine-learning orchestration pipeline pipelines python reverse-etl spark sql transformation
Last synced: 20 Apr 2024
https://github.com/katcosgrove/devops-101-workshop
Serves as documentation, starter code, and companion guide for a DevOps 101 workshop using the JFrog platform.
101 artifactory beginners devops pipelines xray
Last synced: 19 Apr 2024
https://github.com/dmmiller612/sparktorch
Train and run Pytorch models on Apache Spark.
apache-spark deep-learning distributed-computing inference pipelines pytorch sparktorch
Last synced: 19 Apr 2024
https://github.com/intel/dffml
The easiest way to use Machine Learning. Mix and match underlying ML libraries and data set sources. Generate new datasets or modify existing ones with ease.
ai-inference ai-machine-learning ai-training analytics asyncio dag data-flow dataflows datasets dffml event-based flow-based-programming frameworks hyperautomation libraries machine-learning models pipelines python swrepo
Last synced: 16 Apr 2024
https://github.com/izdi/django-slack-oauth
Handles OAuth and stores slack token
django oauth pipelines python registration slack
Last synced: 15 Apr 2024
https://github.com/zenml-io/zenml
ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.
ai automl data-science deep-learning devops-tools hacktoberfest llm llmops machine-learning metadata-tracking ml mlops pipelines production-ready pytorch tensorflow workflow zenml
Last synced: 15 Apr 2024
https://github.com/project-codeflare/codeflare
Simplifying the definition and execution, scaling and deployment of pipelines on the cloud.
automl data-science hyperparameter-optimization machine-learning pipelines ray sklearn workflows
Last synced: 13 Apr 2024
https://github.com/rosineygp/mkdkr
mkdkr = Makefile + Docker
bash cirleci docker github-actions gitlab-ci jenkins makefile pipelines travis-ci
Last synced: 12 Apr 2024
https://github.com/harness/drone-cli
Command Line Tools for Drone CI
continuous-delivery continuous-integration pipelines
Last synced: 10 Apr 2024
https://github.com/hadron-project/hadron
The Kubernetes native and CloudEvents native distributed event streaming, event orchestration & messaging platform
cloudevents dag event-driven events kubernetes messaging orchestration pipelines pubsub rpc rust streams workflow-orchestration workflows
Last synced: 01 Apr 2024
https://github.com/andredesousa/devops-best-practices
This is a guideline of best practices about DevOps.
automated-testing best-practices cicd continuous-delivery continuous-deployment continuous-integration continuous-testing devops infrastructure infrastructure-as-code pipelines
Last synced: 30 Mar 2024
https://github.com/tenzir/tenzir
Open source security data pipelines.
dataops incident-response investigation netflow pcap pipelines secdataops security siem sigma soc suricata threathunting zeek
Last synced: 28 Mar 2024
https://github.com/paysure/orinoco
Functional composable pipelines allowing clean separation of the business logic and its implementation
business-logic business-process fp functional-programming mypy pipelines python3 railway-oriented-programming type-safety
Last synced: 26 Mar 2024
https://github.com/LucianoPAlmeida/variable-injector
Continuous Integration Tool for Swift Projects
ci-pipeline codegen continuous-delivery continuous-integration devops environment environment-variables ios macos pipelines substitutions swift swift-language swift-package-manager values-injected workflow xcode11
Last synced: 24 Mar 2024
https://github.com/substantic/rain
Framework for large distributed pipelines
distributed-computing pipelines python-api rust workflows
Last synced: 24 Mar 2024
https://github.com/JulianKemmerer/PipelineC
A C-like hardware description language (HDL) adding high level synthesis(HLS)-like automatic pipelining as a language construct/compiler feature.
c fpga fpga-acceleration fpga-accelerators fpga-programming hardware hardware-description hardware-description-language high-level-synthesis hls open-source-hardware pipelines python vhdl
Last synced: 21 Mar 2024
https://github.com/intel/ai-workflows
A repository of Dockerfiles, scripts, yaml files, Helm Charts, etc. used to build and scale the sample AI workflows with python, kubernetes, kubeflow, cnvrg.io, and other frameworks on Intel platforms in the cloud and on-premise.
ai ai-workflows cnvrg containerization containers docker docker-compose kubernetes one-api pipelines
Last synced: 17 Mar 2024
https://github.com/metaory/hgit-cli
:rocket: Streamline and Simplify complex processes (a beautiful unopinionated config based workflow engine)
ci ci-cd cli command-line conflict-detection console git git-flow hgit pipeline pipelines script-engine task task-runner terminal tui workflow workflow-automation workflow-engine
Last synced: 15 Mar 2024
