Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/aws/aws-logging-dotnet
.NET Libraries for integrating Amazon CloudWatch Logs with popular .NET logging libraries
https://github.com/aws/aws-logging-dotnet
Last synced: about 2 months ago
JSON representation
.NET Libraries for integrating Amazon CloudWatch Logs with popular .NET logging libraries
- Host: GitHub
- URL: https://github.com/aws/aws-logging-dotnet
- Owner: aws
- License: apache-2.0
- Created: 2016-11-10T21:48:01.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2024-05-22T20:04:14.000Z (4 months ago)
- Last Synced: 2024-05-22T20:50:51.849Z (4 months ago)
- Language: C#
- Size: 4.6 MB
- Stars: 294
- Watchers: 28
- Forks: 134
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# AWS Logging .NET
This repository contains plugins for popular .NET logging frameworks that integrate
with Amazon Web Services. The plugins use the
Amazon CloudWatch Logs service to write log data to a configured
log group. The logs can be viewed and searched using the [AWS CloudWatch Console](https://console.aws.amazon.com/cloudwatch/).
For a history of releases view the [release change log](RELEASE.CHANGELOG.md)
### AWS Lambda
These packages batch logging messages in a queue and send messages to CloudWatch Logs using a background thread.
The use of the background thread means that the messages are not guaranteed to be delivered when used in AWS Lambda.
The reason is because the background thread will be frozen once a Lambda event is processed and
may not ever be unfrozen if more Lambda events are not received for some time.
When using Lambda it is recommended to use either the `ILambdaContext.Logger.LogLine` or the
[Amazon.Lambda.Logging.AspNetCore](https://github.com/aws/aws-lambda-dotnet/tree/master/Libraries/src/Amazon.Lambda.Logging.AspNetCore) package.
### Required IAM Permissions
Regardless of the framework used, the following permissions must be allowed (via [IAM](https://aws.amazon.com/iam)) for the provided AWS credentials.
```
logs:CreateLogGroup
logs:CreateLogStream
logs:PutLogEvents
logs:DescribeLogGroups
```
The practice of granting least privilege access is recommended when setting up credentials. You can further reduce access by limiting permission scope to specific resources (such as a Log Stream) by referencing its ARN during policy creation.
For more information and a sample JSON policy template, please see [Amazon CloudWatch Logs and .NET Logging Frameworks](https://aws.amazon.com/blogs/developer/amazon-cloudwatch-logs-and-net-logging-frameworks/) on the AWS Developer Blog.
### Optional IAM Permissions
The following [IAM](https://aws.amazon.com/iam) permissions are optional depending on the configured features of the logger.
| Feature | IAM Permission(s) for feature | Configuration Setting |
|-------------------------------------------------------------------------------|-------------------------------|-------------------------------|
| [Set new log group retention policy](#setting-new-log-group-retention-policy) | `logs:PutRetentionPolicy` | `NewLogGroupRetentionInDays` |
### Configuring the Log Stream Name
Prior to the versions listed below, these libraries followed CloudWatch Logs'
best practice of having the log stream name be generated. The name could be customized by adding a suffix or prefix using the `LogStreamNameSuffix` and
`LogStreamNamePrefix` configuration properties.
Generating the name ensured that each process within an application has its own log stream to write to. Otherwise when one process writes to the same stream,
the `sequenceToken` maintained within the process goes out of sync. This generated errors and retries causing performance issues.
In 2023 [CloudWatch Logs removed the SequenceToken requirement](https://aws.amazon.com/about-aws/whats-new/2023/01/amazon-cloudwatch-logs-log-stream-transaction-quota-sequencetoken-requirement/),
which removes the need to split log ingestion across multiple log streams
and coordinate the sequence token across multiple clients.
The following versions introduce a new `LogStreamName` setting, which can be used to specify the full log stream name. When this is set `LogStreamNamePrefix` and `LogStreamNameSuffix` will be ignored.
1. AWS.Logger.Nlog - 3.3.0
2. AWS.Logger.Log4net - 3.5.0
3. AWS.Logger.AspNetCore - 3.5.0
4. AWS.Logger.SeriLog - 3.4.0
### Setting new Log Group Retention Policy
These libraries support setting a [log retention policy](https://docs.aws.amazon.com/managedservices/latest/userguide/log-customize-retention.html)
on any CloudWatch Log Groups which they create. This feature is enabled using the NewLogGroupRetentionInDays configuration property.
The DisableLogGroupCreation configuration property must not be set to true. Retention policies configured in this manner
are only applied to _new_ Log Groups created directly by these libraries. By default no retention policy is applied to newly created Log Groups.
Note that any value of NewLogGroupRetentionInDays which is not one supported by CloudWatch [which can be found here](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutRetentionPolicy.html#API_PutRetentionPolicy_RequestSyntax) - and listed below - is a configuration error which will result
in a non-fatal error applying the policy. The application and logging will continue however no retention policy will be applied.
```csharp
null, 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, 3653
```
## Supported Logging Frameworks
1. [NLog](#nlog)
2. [Apache log4net](#apache-log4net)
3. [ASP.NET Core Logging](#aspnet-core-logging)
4. [Serilog](#serilog)
### NLog
* NuGet Package: [AWS.Logger.Nlog](https://www.nuget.org/packages/AWS.Logger.NLog/)
NLog uses targets that can be configured to receive log messages. Targets can be configured either
through a config file or through code. The default config file that NLog will automatically search for
is **NLog.config**. Here is an example config file that configures the AWS Region and the CloudWatch Logs log group.
```xml
```
The AWS credentials will be found using the standard AWS SDK for .NET credentials search path. In this case
it will look for a profile named default, search for environment variables or search for an instance profile on an
EC2 instance. To use a specific AWS credential profile use the **profile** attribute on the target.
Here is an example of performing the same configuration via code.
```csharp
var config = new LoggingConfiguration();
var awsTarget = new AWSTarget()
{
LogGroup = "NLog.ProgrammaticConfigurationExample",
Region = "us-east-1"
};
config.AddTarget("aws", awsTarget);
config.LoggingRules.Add(new LoggingRule("*", LogLevel.Debug, awsTarget));
LogManager.Configuration = config;
```
Checkout the [NLog samples](/samples/NLog) for examples on how you can use AWS and NLog together.
### Apache log4net
* NuGet Package: [AWS.Logger.Log4net](https://www.nuget.org/packages/AWS.Logger.Log4net/)
Log4net configures appenders to receive log messages. Appenders can be configured either
through a config file or through code. To use a config file add a file to your project.
The file can be named anything but for this example call it log4net.config. Make sure
that **Copy to Output Directory** is set to copy. Here is an example config file setting
the CloudWatch Log log group and the AWS Region.
```xml
Log4net.ConfigExample
us-east-1
```
The AWS credentials will be found using the standard AWS SDK for .NET credentials search path. In this case
it will look for a profile named default, search for environment variables or search for an instance profile on an
EC2 instance. To use a specific AWS credential profile add a **Profile** under the **appender** node.
Add the following code during the startup of the application to have log4net read the configuration file.
```
// log4net is configured in the log4net.config file which adds the AWS appender.
XmlConfigurator.Configure(new System.IO.FileInfo("log4net.config"));
```
Here is an example of performing the same configuration via code.
```csharp
static void ConfigureLog4net()
{
Hierarchy hierarchy = (Hierarchy)LogManager.GetRepository();
PatternLayout patternLayout = new PatternLayout();
patternLayout.ConversionPattern = "%-4timestamp [%thread] %-5level %logger %ndc - %message%newline";
patternLayout.ActivateOptions();
AWSAppender appender = new AWSAppender();
appender.Layout = patternLayout;
// Set log group and region. Assume credentials will be found using the default profile or IAM credentials.
appender.LogGroup = "Log4net.ProgrammaticConfigurationExample";
appender.Region = "us-east-1";
appender.ActivateOptions();
hierarchy.Root.AddAppender(appender);
hierarchy.Root.Level = Level.All;
hierarchy.Configured = true;
}
```
Checkout the [Log4net samples](/samples/Log4net) for examples of how you can use AWS and log4net together.
### ASP.NET Core Logging
* NuGet Package: [AWS.Logger.AspNetCore](https://www.nuget.org/packages/AWS.Logger.AspNetCore/)
ASP.NET Core introduced a new [logging framework](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging) that has providers configured to send logs to destinations.
The AWS.Logger.AspNetCore NuGet package provides a log provider which adds CloudWatch Logs as a destination for the logs.
**Note:** Starting with version 2.0.0 of AWS.Logger.AspNetCore this library targets netstandard2.0 and the dependencies have been
upgraded to the ASP.NET Core 2.1 versions. For older versions of .NET Core, which Microsoft has made end of life, use versions before 2.0.0.
The [WebSample](/samples/AspNetCore/WebSample) in this repository demonstrates how to configure
this provider.
The configuration is setup in the [appsettings.json](/samples/AspNetCore/WebSample/appsettings.json) file. In versions before 2.0.0 the `AWS.Logging`
was used as the configuration section root. Starting with 2.0.0 the library has switched to use the standard `Logging` configuration section root.
For backwards compatibility if the `Logging` section does not contain a `LogGroup` then the library will fallback to `AWS.Logging`.
```json
"Logging": {
"Region": "us-east-1",
"LogGroup": "AspNetCore.WebSample",
"IncludeLogLevel": true,
"IncludeCategory": true,
"IncludeNewline": true,
"IncludeException": true,
"IncludeEventId": false,
"IncludeScopes": false,
"LogLevel": {
"Default": "Debug",
"System": "Information",
"Microsoft": "Information"
}
}
```
In a typical ASP.NET Core application the `Program.cs` file contains a `CreateWebHostBuilder` method. To include AWS.Logger.AspNetCore
add a call to `ConfigureLogging` and in the `Action` passed into ConfigureLogging call `AddAWSProvider`. This will look up the configuration
information from the IConfiguration added to the dependency injection system.
```csharp
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureLogging(logging =>
{
logging.AddAWSProvider();
// When you need logging below set the minimum level. Otherwise the logging framework will default to Informational for external providers.
logging.SetMinimumLevel(LogLevel.Debug);
})
.UseStartup();
```
### Serilog
* NuGet Package: [AWS.Logger.SeriLog](https://www.nuget.org/packages/AWS.Logger.SeriLog/)
Serilog can be configured with sinks to receive log messages either through a config file or through code. To use a config file with Serilog, follow the instructions [here](https://github.com/serilog/serilog/wiki/Configuration-Basics)
to install the necessary extensions and NuGet packages. In the json file, make sure **AWS.Logger.SeriLog** is in the **Using**
array. Set the **LogGroup** and **Region** under the **Serilog** node, and add **AWSSeriLog** as a sink under the **WriteTo** node. Here is an example.
```json
{
"Serilog": {
"Using": [
"AWS.Logger.SeriLog"
],
"LogGroup": "Serilog.ConfigExample",
"Region": "us-east-1",
"MinimumLevel": "Information",
"WriteTo": [
{
"Name": "AWSSeriLog"
}
]
}
}
```
Add the following code to configure the logger to read from the json file.
```csharp
var configuration = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.Build();
var logger = new LoggerConfiguration()
.ReadFrom.Configuration(configuration)
.CreateLogger();
```
The AWS Credentials will be found using the standard .NET credentials search path. It will search for a profile named default, environment variables, or an instance profile on an EC2 instance.
In order to use a profile other than default, add a **Profile** under the **Serilog** node.
Below is an example of doing the same configuration as above via code. The AWS sink can be added to the logger by using the WriteTo
method.
```csharp
AWSLoggerConfig configuration = new AWSLoggerConfig("Serilog.ConfigExample");
configuration.Region = "us-east-1";
var logger = new LoggerConfiguration()
.WriteTo.AWSSeriLog(configuration)
.CreateLogger();
```
Checkout the [Serilog samples](/samples/Serilog) for examples of how you can use AWS and Serilog together.