Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/thomhurst/ModularPipelines
Write your pipelines in C# !
https://github.com/thomhurst/ModularPipelines
build build-system build-tool build-tools csharp dotnet dotnet-core dotnetcore pipeline pipeline-deploy pipeline-framework pipelines
Last synced: about 1 month ago
JSON representation
Write your pipelines in C# !
- Host: GitHub
- URL: https://github.com/thomhurst/ModularPipelines
- Owner: thomhurst
- License: mit
- Created: 2023-05-22T22:19:01.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-04-10T22:27:06.000Z (about 2 months ago)
- Last Synced: 2024-04-11T18:21:12.888Z (about 2 months ago)
- Topics: build, build-system, build-tool, build-tools, csharp, dotnet, dotnet-core, dotnetcore, pipeline, pipeline-deploy, pipeline-framework, pipelines
- Language: C#
- Homepage: https://thomhurst.github.io/ModularPipelines/
- Size: 39.7 MB
- Stars: 167
- Watchers: 2
- Forks: 4
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: License.txt
Lists
- awesome-dotnet - ModularPipelines - Write your pipelines in C# (Build Automation)
README
# ModularPipelines
Define your pipeline in .NET! Strong types, intellisense, parallelisation, and the entire .NET ecosystem at your fingertips.
[](https://www.nuget.org/packages/ModularPipelines/)
   [](https://app.codacy.com/gh/thomhurst/ModularPipelines/dashboard?utm_source=gh\&utm_medium=referral\&utm_content=\&utm_campaign=Badge_grade) [](https://www.codefactor.io/repository/github/thomhurst/modularpipelines)  [](https://app.codacy.com/gh/thomhurst/ModularPipelines/dashboard?utm_source=gh\&utm_medium=referral\&utm_content=\&utm_campaign=Badge_coverage) [](https://codecov.io/gh/thomhurst/ModularPipelines)
## [Documentation](https://thomhurst.github.io/ModularPipelines)
## Features
* Parallel execution
* Dependency management
* Familiar C# code
* Ability to debug pipelines
* Ability to run pipelines locally, even creating versions for setting up local development
* Strong typing, where different modules/steps can pass data to one another
* Dependency collision detection - Don't worry about accidentally making two modules dependent on each other
* Numerous helpers to do things like: Search files, check checksums, (un)zip folders, download files, install files, execute CLI commands, hash data, and more
* Easy to Skip or Ignore Failures for each individual module by passing in custom logic
* Hooks that can run before and/or after modules
* Pipeline requirements - Validate your requirements are met before executing your pipeline, such as a Linux operating system
* Easy to use File and Folder classes, that can search, read, update, delete and more
* Source controlled pipelines
* Build agent agnostic - Can easily move to a different build system without completely recreating your pipeline
* No need to learn new syntaxes such as YAML defined pipelines
* Strongly typed wrappers around command line tools
* Utilise existing .NET libraries
* Secret obfuscation
* Grouped logging, and the ability to extend sources by adding to the familiar `ILogger`
* Run based on categories
* Easy to read exceptions
* Dynamic console progress reporting (if the console supports interactive mode)
* Pretty results table
## Available Modules
| Package | Description | Version |
| --- | --- | --- |
| ModularPipelines | Write your pipelines in C#! | [](https://www.nuget.org/packages/ModularPipelines/) |
| ModularPipelines.AmazonWebServices | Helpers for interacting with Amazon Web Services. | [](https://www.nuget.org/packages/ModularPipelines.AmazonWebServices/) |
| ModularPipelines.Azure | Helpers for interacting with Azure. | [](https://www.nuget.org/packages/ModularPipelines.Azure/) |
| ModularPipelines.Azure.Pipelines | Helpers for interacting with Azure Pipeline agents. | [](https://www.nuget.org/packages/ModularPipelines.Azure.Pipelines/) |
| ModularPipelines.Chocolatey | Helpers for interacting with the Chocolatey CLI. | [](https://www.nuget.org/packages/ModularPipelines.Chocolatey/) |
| ModularPipelines.Cmd | Helpers for interacting with the Windows cmd process. | [](https://www.nuget.org/packages/ModularPipelines.Cmd/) |
| ModularPipelines.Docker | Helpers for interacting with the Docker CLI. | [](https://www.nuget.org/packages/ModularPipelines.Docker/) |
| ModularPipelines.DotNet | Helpers for interacting with dotnet CLI. | [](https://www.nuget.org/packages/ModularPipelines.DotNet/) |
| ModularPipelines.Email | Helpers for sending emails. | [](https://www.nuget.org/packages/ModularPipelines.Email/) |
| ModularPipelines.Ftp | Helpers for downloading and uploading via FTP. | [](https://www.nuget.org/packages/ModularPipelines.Ftp/) |
| ModularPipelines.Git | Helpers for interacting with git. | [](https://www.nuget.org/packages/ModularPipelines.Git/) |
| ModularPipelines.GitHub | Helpers for interacting with GitHub Actions build agents. | [](https://www.nuget.org/packages/ModularPipelines.GitHub/) |
| ModularPipelines.Helm | Helpers for interacting with Helm CLI. | [](https://www.nuget.org/packages/ModularPipelines.Helm/) |
| ModularPipelines.Kubernetes | Helpers for interacting with kubectl CLI. | [](https://www.nuget.org/packages/ModularPipelines.Kubernetes/) |
| ModularPipelines.MicrosoftTeams | Helpers for sending Microsoft Teams cards. | [](https://www.nuget.org/packages/ModularPipelines.MicrosoftTeams/) |
| ModularPipelines.Node | Helpers for interacting with node / npm CLI. | [](https://www.nuget.org/packages/ModularPipelines.Node/) |
| ModularPipelines.NuGet | Helpers for interacting with dotnet nuget CLI. | [](https://www.nuget.org/packages/ModularPipelines.NuGet/) |
| ModularPipelines.Slack | Helpers for sending Slack cards. | [](https://www.nuget.org/packages/ModularPipelines.Slack/) |
| ModularPipelines.TeamCity | Helpers for interacting with TeamCity build agents. | [](https://www.nuget.org/packages/ModularPipelines.TeamCity/) |
| ModularPipelines.Terraform | Helpers for interacting with Terraform CLI. | [](https://www.nuget.org/packages/ModularPipelines.Terraform/) |
| ModularPipelines.WinGet | Helpers for interacting with the Windows Package Manager. | [](https://www.nuget.org/packages/ModularPipelines.WinGet/) |
| ModularPipelines.Yarn | Helpers for interacting with Yarn CLI. | [](https://www.nuget.org/packages/ModularPipelines.Yarn/) |
## Getting Started
If you want to see how to get started, or want to know more about ModularPipelines, [read the Documentation here](https://thomhurst.github.io/ModularPipelines)
## Console Progress
## Results
## How does this compare to Cake / Nuke
* Strong types! You have complete control over what data, and what shape of data to pass around from and to different modules
* No external tooling is required. Pipelines are run with a simple `dotnet run`
* Full dependency injection support for your services
* Similar and familiar setup to frameworks like ASP.NET Core
* Real C# - Whereas frameworks like cake are a scripted form of C#
* Parallelism - Work will run concurrently unless it is dependent on something else
* The style of writing pipelines is very different - Work is organised into separate module classes, keeping code organised and more closely following SRP than having all your work in one main class. This also helps multiple contributors avoid things like merge conflicts
## Code Examples
### Program.cs - Main method
```csharp
await PipelineHostBuilder.Create()
.ConfigureAppConfiguration((context, builder) =>
{
builder.AddJsonFile("appsettings.json")
.AddUserSecrets()
.AddEnvironmentVariables();
})
.ConfigureServices((context, collection) =>
{
collection.Configure(context.Configuration.GetSection("NuGet"));
collection.Configure(context.Configuration.GetSection("Publish"));
collection.AddSingleton();
collection.AddTransient();
})
.AddModule()
.AddModule()
.ExecutePipelineAsync();
```
### Custom Modules
```csharp
public class FindNugetPackagesModule : Module
{
protected override Task?> ExecuteAsync(IPipelineContext context, CancellationToken cancellationToken)
{
return context.Git()
.RootDirectory
.GetFiles(path => path.Extension is ".nupkg")
.ToList()
.AsTask();
}
}
```
```csharp
[DependsOn]
public class UploadNugetPackagesModule : Module
{
private readonly IOptions _nugetSettings;
public UploadNugetPackagesModule(IOptions nugetSettings)
{
_nugetSettings = nugetSettings;
}
protected override async Task ExecuteAsync(IPipelineContext context, CancellationToken cancellationToken)
{
var nugetFiles = await GetModule();
return await context.NuGet()
.UploadPackages(new NuGetUploadOptions(nugetFiles.Value!.AsPaths(), new Uri("https://api.nuget.org/v3/index.json"))
{
ApiKey = _nugetSettings.Value.ApiKey
});
}
}
```
### Breaking changes
While I will try to limit breaking changes, there may be some changes within minor versions. These will be noted on release notes.