Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with data-factory

A curated list of projects in awesome lists tagged with data-factory .

https://github.com/benahm/TestDataFactory

The ultimate Apex Test Data Factory 🏭

apex data-factory salesforce

Last synced: 06 May 2025

https://github.com/mrpaulandrew/procfwk

A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.

adf adfprocfwk azure azure-functions azure-sql-database data-engineering data-factory framework metadata pipelines processing procfwk

Last synced: 08 Apr 2025

https://github.com/microsoft/data-factory-testing-framework

A stand-alone test framework that allows to write unit tests for Data Factory pipelines on Microsoft Fabric, Azure Data Factory and Azure Synapse Analytics.

azure-data-factory azure-synapse data-factory fabric functional-tests microsoft-fabric test testing-framework unit-tests

Last synced: 12 Apr 2025

https://github.com/houbb/data-factory

🏭Auto generate mock data for java test.(便于 Java 测试自动生成对象信息)

data-factory data-prepare java-data-builder junit-test mock mock-data mock-test

Last synced: 10 Apr 2025

https://github.com/azure/data-factory-deploy-action

GitHub Action for side-effect free deployment of Azure Data Factory resources

arm-templates azure data-factory deployment github-actions

Last synced: 25 Jan 2025

https://github.com/mrpaulandrewltd/Microsoft-Data-Integration-Pipeline-Training

Training workshop content on Azure Data Factory and Azure Synapse Analytics Data Integration Pipelines

azure data data-factory integration pipelines procfwk synapse-analytics

Last synced: 31 Mar 2025

https://github.com/lren-chuv/airflow-imaging-plugins

Set of plugins helping to work with imaging data in Airflow.

airflow data-factory mri neuroimaging neuroscience spm

Last synced: 13 May 2025

https://github.com/rubensworks/rdf-data-factory.js

A TypeScript/JavaScript implementation of the RDF/JS data factory.

data-factory data-model hacktoberfest rdf rdfjs

Last synced: 22 Mar 2025

https://github.com/azure/data-factory-export-action

GitHub Action that exports Azure Data Factory resources to an ARM Template

azure data-factory github-actions

Last synced: 25 Jan 2025

https://github.com/azure/data-factory-validate-action

GitHub Action that validates all of the Azure Data Factory resources in the Git repository

azure data-factory github-actions

Last synced: 20 Nov 2024

https://github.com/lren-chuv/data-tracking

This project contains scripts to extract metadata from DICOM files, NIFTI files and spreadsheets, and import them into a database.

data-factory dicom nifti python

Last synced: 13 May 2025

https://github.com/lren-chuv/data-factory-airflow-dags

DAGs adapting the MRI preprocessing pipeline to Airflow

airflow data-factory mri neuroimaging nifti

Last synced: 13 May 2025

https://github.com/thunchanokbow/walmart-end-to-end-with-azure-databricks

The data is extracted in batches using Azure Data Factory. and transformed using Spark in Azure Databricks. The transformed data is stored in Synapse Analytics in columnar format. Finally, a dashboard is created using Looker Studio or Power BI to display sales data, customer analysis, and other information

dashboard data-factory databricks gen2 synapse-analytics walmart

Last synced: 26 Feb 2025

https://github.com/meken/sql-partitioned-tables

Showcase on how to do partitioned tables on Azure SQL Databases with sliding windows

archival azure data-factory partitioning sql

Last synced: 16 May 2025

https://github.com/lren-chuv/i2b2-import

This library provides functions to import data into an I2B2 DB schema

data-factory i2b2 pytthon-3

Last synced: 13 May 2025

https://github.com/lren-chuv/mri-parallel-nmm-pipeline

Wrapper to run the SPM12 neuromorphometric pipeline in parallel over multiple CPU cores

data-factory

Last synced: 12 Mar 2025

https://github.com/lren-chuv/data-catalog-setup

Migration scripts for the Data Catalog database used to capture provenance information on all files

data-factory database-migrations docker-image

Last synced: 12 Mar 2025

https://github.com/lren-chuv/docker-cbrain

Docker images for CBrain

data-factory docker-images

Last synced: 12 Mar 2025

https://github.com/omaciasd/datafactory-migration

Guide for Data Factory migration to a new tenant.

arm azure azure-devops data-factory python

Last synced: 24 Mar 2025

https://github.com/lren-chuv/i2b2-flattening

Generate a flat CSV file from an I2B2 data (containing brain features, demographic data, etc).

data-factory

Last synced: 12 Mar 2025

https://github.com/pandabear-neil/microsoft_fabric_gems

Code Snippets, designs, and thinkings around the Microsoft Fabric Platform

data-engineering data-factory data-science data-warehouse microsoft-fabric spark

Last synced: 03 Apr 2025

https://github.com/lren-chuv/hierarchizer

Reorganize DICOM files following a hierarchy based on meta-data fields

data-factory dicom docker-image mri python-3

Last synced: 12 Mar 2025

https://github.com/lren-chuv/nmm-pipeline-nifti-organizer

This tool is meant to help data owners to organise their MRI data in such way that it can be pre-processed by the MIP SPM12 neuromorphometric pipeline as wrapped for the MIP.

data-factory

Last synced: 12 Mar 2025

https://github.com/lren-chuv/ehr-to-i2b2

Import EHR data to I2B2.

data-factory docker-image etl i2b2

Last synced: 12 Mar 2025

https://github.com/lren-chuv/nmm-pipeline-output-merge

Merge individual MIP formatted output from the NMM pipeline into one single CSV file.

data-factory

Last synced: 12 Mar 2025

https://github.com/lren-chuv/labkey-docker

Docker container for LabKey.

data-factory docker-image

Last synced: 12 Mar 2025

https://github.com/yosrak5/predictive_maintenance

End-To-End Predictive Analytics to predict Hardware Maintenance Before it occurs using data preprocessing and Machine Learning classification modeling (XGBOOST , Random Forest )

data-factory data-perspective machine-learning-workbench matplotlib numpy pandas python random-forest seaborn xgboost

Last synced: 09 Apr 2025