{"id":28089083,"url":"https://github.com/erwinkramer/otel-business","last_synced_at":"2025-08-22T13:34:12.578Z","repository":{"id":291236409,"uuid":"969964618","full_name":"erwinkramer/otel-business","owner":"erwinkramer","description":"Get started with distributed business tracing in context of OTel (OpenTelemetry).","archived":false,"fork":false,"pushed_at":"2025-05-21T08:38:23.000Z","size":5800,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-24T23:11:47.162Z","etag":null,"topics":["application-insights","azure","azure-application-insights","azure-monitor","bam","business-activity-monitor","open-telemetry","opentelemetry","otel"],"latest_commit_sha":null,"homepage":"https://www.guanchen.nl","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/erwinkramer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-21T08:21:27.000Z","updated_at":"2025-05-27T08:22:31.000Z","dependencies_parsed_at":"2025-05-03T09:54:06.870Z","dependency_job_id":"0bdd5a5f-a40f-494e-bfba-771a069d8bf7","html_url":"https://github.com/erwinkramer/otel-business","commit_stats":null,"previous_names":["erwinkramer/otel-business"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/erwinkramer/otel-business","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erwinkramer%2Fotel-business","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erwinkramer%2Fotel-business/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erwinkramer%2Fotel-business/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erwinkramer%2Fotel-business/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/erwinkramer","download_url":"https://codeload.github.com/erwinkramer/otel-business/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erwinkramer%2Fotel-business/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262838507,"owners_count":23372544,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["application-insights","azure","azure-application-insights","azure-monitor","bam","business-activity-monitor","open-telemetry","opentelemetry","otel"],"created_at":"2025-05-13T12:53:38.043Z","updated_at":"2025-06-30T19:33:12.267Z","avatar_url":"https://github.com/erwinkramer.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Business tracing with OTel 💼\n\n[![CC BY-NC-SA 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa]\n![GitHub commit activity](https://img.shields.io/github/commit-activity/m/erwinkramer/otel-business)\n\n![Business tracing with OTel mascot](.images/mascot-small.png)\n\nGet started with distributed business tracing in context of OTel (OpenTelemetry).\n\n[Guanchen.Monitor](/Guanchen.Monitor/) is the component library to facilitate business tracing. Currently supported usages are:\n\n1. Console Apps, sample via [IntegrationTest.Console](/IntegrationTest.Console/)\n1. Function Apps, sample via [IntegrationTest.Function](/IntegrationTest.Function/)\n\nThe main features (extended on top of a default OpenTelemetry setup) are:\n\n| Functionality | Description\n|--|--\n| StartBusinessActivity (Parent/Child/Linked) methods | Starts a standardized business activity (parent, child, or linked).\n| NewBusinessEvent method | Creates a standardized business event.\n| LogBusiness (Information/Error) methods | Creates a standardized business log.\n| Standardization of OTel Baggage | All business activities, events and logs contain the same business identifiers such as the `BusinessTrace`-tag and the 💼-prefix and continuously will be set (along with other OTel Baggage) on business activities, events and logs.\n| Standardization of KQL analysis | Standardized Azure Monitor KQL-queries for business activities, events and logs.\n| Reliable processing of activities and events | `StartBusinessActivity` and `NewBusinessEvent` are reliably handled via the `AutoFlushActivityProcessor` and suitable for auditing purposes.\n\n## Library positioning\n\nBusiness tracing in this project simply sits right between your app and the [Azure Monitor OpenTelemetry Distro](https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-enable?tabs=aspnetcore), as shown below:\n\n```mermaid\nflowchart TB\n\nfunc[\"Azure Function App\"]\nasp[\"ASP.NET Core App\"]\ncons[\"Console App\"]\n\ngm[\"Business tracing with Guanchen.Monitor\"]\n\nsubgraph azm[Azure Monitor]\nazmdistro[OpenTelemetry Distro]\nazmappi[Application Insights]\nazmtables[Traces/Requests/Exceptions Tables]\nend\n\nfunc --\u003e gm\nasp --\u003e gm\ncons --\u003e gm\ngm -- implements --\u003e azmdistro\nazmdistro -- exports with OpenTelemetry Protocol (OTLP), to --\u003e azmappi\nazmappi -- persists in --\u003e azmtables\n\n```\n\n## Usage\n\n### Spans\n\n\u003e ℹ️ [OTel Spans](https://opentelemetry.io/docs/concepts/signals/traces/#spans) are [Activities in .NET](https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.activity) and end up in the `requests` table.\n\n\u003e ℹ️ [OTel Trace IDs](https://opentelemetry.io/docs/concepts/signals/traces/#span-context) are [ActivityTraceId's in .NET](https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.activitytraceid) and [result](https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-overview#opentelemetry) in [operation IDs](https://learn.microsoft.com/en-us/azure/azure-monitor/app/data-model-complete#:~:text=operation_Id-,OperationId,-The%20unique%20identifier).\n\nUse `StartParentBusinessActivity()` to start a business span and `StartLinkedBusinessActivity()` to start a business span linked to another span. Both of these methods generate new operation IDs. These operation IDs are an important tool to analyze business logging and can effectively seen as a trace of business operations.\n\nUse `StartChildBusinessActivity()` to start a child business span. This method will not generate a new operation ID and is generally suited for sub-processes within a business operation.\n\n### Span Events\n\n\u003e ℹ️ [OTel Span Events](https://opentelemetry.io/docs/concepts/signals/traces/#span-events) are [ActivityEvents in .NET](https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.activityevent) and end up in the `traces` table.\n\nUse `NewBusinessEvent()` to create business events within a span. These events are stored in the same table as [Logs](#logs) but are directly associated with a span, ensuring more reliable delivery. For more details, refer to the [Reliability notes](#reliability-notes).\n\n### Logs\n\n\u003e ℹ️ [OTel Logs](https://opentelemetry.io/docs/concepts/signals/logs/) are [ILogger logs in .NET](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.ilogger) and end up in the `traces` table.\n\nUse `LogBusinessInformation()` or `LogBusinessError()` to create business logs within a span.\n\n### Baggage\n\n\u003e ℹ️ [OTel Baggage](https://opentelemetry.io/docs/concepts/signals/baggage/) keeps contextual information and [propagates](https://opentelemetry.io/docs/concepts/context-propagation/#propagation) the information (which currently has its limits, as explained at [the Caveats paragraph](#caveats)). Technically, on .NET Activities they are `Tags` and for .NET ILogger logs they will be set on `OpenTelemetry.Logs.LogRecord.Attributes`. For both they will end up in the [Custom properties](https://learn.microsoft.com/en-us/azure/azure-monitor/app/data-model-complete#custom-properties-1) column of their related Azure Monitor table (`requests` and `traces` respectively).\n\nUse `Baggage.SetBaggage()` to set business context information to the root span, this persists throughout other spans (and their logs and events) that use the same root span, unless overwritten by a more recent `Baggage.SetBaggage()`.\n\nUse `yourActivity.SetBaggage()` to set business context information to an Activity, this persists throughout other child spans (and their logs and events).\n\nThis project makes sure that the baggage is continuously being set on business logs, business spans and business span events to make this information available in Azure Monitor.\n\nLog, span and span event functions implicitly yield a `Business Trace` baggage key with the level (`Information`, `Error` etc.) als value.\n\n## Integration tests\n\nThe integration test has the following OTel Span/Activity setup. There are 4 unique activities in total, 1 for the complete batch; `Splitting Tomato Batch`, and 3 for each tomato in the batch; `Evaluating Tomato`, `Auditing HTTP Tomato` and `Auditing Queue Tomato`. All activities are related to each other, either as link or as child.\n\nIn the following diagram the activities are related to the operation IDs:\n\n```mermaid\nflowchart TD\n\nact_evaluating[\"Evaluating Tomato\"]\nact_audit_http[\"Auditing HTTP Tomato\"]\nact_audit_queue[\"Auditing Queue Tomato\"]\n\nsubgraph op_parent[\"Unique batch operation ID\"]\n\nact_splitting[\"Splitting Tomato Batch\"]\n\nend\n\nsubgraph op_child[\"Unique child operation ID\"]\n\nact_splitting -- has linked activity --\u003e act_evaluating\nact_splitting -- has linked activity --\u003e act_evaluating\nact_splitting -- has linked activity --\u003e act_evaluating\n\nact_evaluating -- has child activity --\u003e act_audit_http\nact_evaluating -- has child activity --\u003e act_audit_queue\n\nend\n```\n\nIn the following diagram the activities are related to the hosting resources:\n\n```mermaid\nflowchart TB\n\nact_evaluating[\"Evaluating Tomato\"]\nact_audit_http[\"Auditing HTTP Tomato\"]\nact_audit_queue[\"Auditing Queue Tomato\"]\n\nsubgraph op_parent[\"Console App\"]\n\nact_splitting -- has linked activity --\u003e act_evaluating\nact_splitting -- has linked activity --\u003e act_evaluating\nact_splitting -- has linked activity --\u003e act_evaluating\n\nact_splitting[\"Splitting Tomato Batch\"]\n\nend\n\nsubgraph resource_sb[\"Service Bus\"]\nqueue[📬]\n\nact_evaluating -- passing through --\u003e queue\n\nend\n\nsubgraph op_child[\"Function App\"]\n\nqueue -- has child activity --\u003e act_audit_queue\nact_evaluating -- has child activity --\u003e act_audit_http\n\nend\n```\n\n### General prerequisites\n\nDeploy and configure an Azure Service Bus with, as it's being used to send messages via the Console App to the Function App.\n\n### IntegrationTest.Console prerequisites\n\nFor [IntegrationTest.Console](/IntegrationTest.Console/), prepare the following:\n\n1. Set the Environment Variables on your machine:\n\n    ```cmd\n    setx APPLICATIONINSIGHTS_CONNECTION_STRING \"InstrumentationKey=...;IngestionEndpoint=...;LiveEndpoint=...;ApplicationId=...\"\n    setx AZURE_TENANT_ID \"...\"\n    setx AZURE_SERVICEBUS_FULLYQUALIFIEDNAMESPACE \"....servicebus.windows.net\"\n    ```\n\n1. Set the permissions for the identity running the Console App (likely yourself) on the Azure Service Bus to write messages.\n\n### IntegrationTest.Function prerequisites\n\nFor [IntegrationTest.Function](/IntegrationTest.Function/), prepare the following:\n\n1. Provision a Function App resource and deploy the Function App.\n1. Set the permissions for the identity running the Function App (likely the Managed Identity) on the Azure Service Bus to read messages.\n1. Configure the App Insights connection and the Service Bus connection (`ServiceBusConnection`). The environment variables inside your Function App resource should look similar to this:\n\n    ```json\n    [\n        {\n            \"name\": \"APPLICATIONINSIGHTS_CONNECTION_STRING\",\n            \"value\": \"InstrumentationKey=...;IngestionEndpoint=...;LiveEndpoint=...;ApplicationId=...\",\n            \"slotSetting\": false\n        },\n        {\n            \"name\": \"AzureWebJobsStorage\",\n            \"value\": \"DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net\",\n            \"slotSetting\": false\n        },\n        {\n            \"name\": \"DEPLOYMENT_STORAGE_CONNECTION_STRING\",\n            \"value\": \"DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net\",\n            \"slotSetting\": false\n        },\n        {\n            \"name\": \"ServiceBusConnection__credential\",\n            \"value\": \"managedidentity\",\n            \"slotSetting\": false\n        },\n        {\n            \"name\": \"ServiceBusConnection__fullyQualifiedNamespace\",\n            \"value\": \"....servicebus.windows.net\",\n            \"slotSetting\": false\n        }\n    ]\n    ```\n\n## Analyzing via Transaction view\n\nUse [Transaction Diagnostics](https://learn.microsoft.com/en-us/azure/azure-monitor/app/transaction-search-and-diagnostics?tabs=transaction-diagnostics) to find the business traces and requests intertwined with the full technical diagnostics. It might look like this:\n\n![Transaction View](.images/transactionview.png)\n\n## Analyzing via query\n\nUse a kql query to get all business information. It might look like this:\n\n![Transaction View](.images/queryview.png)\n\nTo replicate this query result, do the following:\n\nCreate [function](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/cross-workspace-query#query-by-using-a-function) `AppRequestsTomatoScope` with all the distributed request tables:\n\n```kql\nunion withsource= SourceApp\nworkspace(\"6d489906-8b6a-4eba-b697-373f29a1a98b\").AppRequests\n```\n\nCreate [function](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/cross-workspace-query#query-by-using-a-function)  `AppTracesTomatoScope` with all the distributed trace tables:\n\n```kql\nunion withsource= SourceApp\nworkspace(\"6d489906-8b6a-4eba-b697-373f29a1a98b\").AppTraces\n```\n\nRun the following query to figure out what operation IDs are related to specific Tomato IDs:\n\n```kql\nlet TomatoIds = dynamic([\n    \"1c946279-04b9-4c25-8846-0dabf1d59533\"\n]);\nAppRequestsTomatoScope\n| extend \n    TomatoId = tostring(Properties[\"Tomato ID\"])\n| where TomatoId in (TomatoIds)\n| distinct OperationId, TomatoId\n```\n\nRun the following query to analyze specific operation IDs:\n\n```kql\nlet OperationIds = dynamic([\n    \"db58273aca2e0e93a3ca4897a99ab36a\"\n]);\nlet Requests = AppRequestsTomatoScope\n    | where OperationId in (OperationIds)\n    | project BusinessServiceSpanId = Id, BusinessService = Name;\nAppTracesTomatoScope\n| where OperationId in (OperationIds)\n| extend \n    BusinessTrace = tostring(Properties[\"Business Trace\"]),\n    TomatoId = tostring(Properties[\"Tomato ID\"]),\n    TomatoPartId = tostring(Properties[\"Tomato Part ID\"])\n| where isnotempty(BusinessTrace)\n| project\n    ParentId,\n    TimeGenerated,\n    BusinessTrace,\n    Message,\n    TomatoId,\n    TomatoPartId\n| join kind=leftouter (Requests) on $left.ParentId == $right.BusinessServiceSpanId\n| project\n    TimeGenerated,\n    BusinessTrace,\n    BusinessService,\n    BusinessServiceSpanId,\n    Message,\n    TomatoId,\n    TomatoPartId\n| order by BusinessService asc, TimeGenerated asc\n```\n\n## Target tables\n\nThe following table mappings are relevant in this project:\n\n| Azure Monitor Table | OpenTelemetry DataType | .NET Implementation\n|---|---|---\n| customMetrics        | Metrics                         | System.Diagnostics.Metrics.Meter\n| exceptions           | Exceptions                      | System.Exception\n| requests             | Spans (Server, Producer)        | System.Diagnostics.Activity\n| traces               | Logs                            | Microsoft.Extensions.Logging.ILogger\n| traces               | Span Events                     | System.Diagnostics.ActivityEvent\n\nFor a complete overview, see [How do Application Insights telemetry types map to OpenTelemetry?](https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-dotnet-migrate?tabs=aspnetcore#how-do-application-insights-telemetry-types-map-to-opentelemetry).\n\n## Loglevel\n\n### IntegrationTest.Console\n\n`CreateLoggerFactory` contains `SetMinimumLevel`.  \n\n### IntegrationTest.Function\n\n`host.json` contains `logLevel.default`.\n\n## Reliability notes\n\n### Logs\n\n\u003e ❌ Using logs for auditing purposes is not supported.\n\n**Eventual export of logs** to Azure Monitor is made possible by [Offline Storage and Automatic Retries](https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-configuration?tabs=net#offline-storage-and-automatic-retries), which are enabled by default in the [Azure Monitor OpenTelemetry Distro](https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-enable?tabs=aspnetcore), but [isn't a feature of the base .NET OpenTelemetry implementation yet](https://github.com/open-telemetry/opentelemetry-dotnet/issues/4115).\n\n1. This doesn't reliably send messages again in case of a disaster at the client side, only when there is an outage at the Azure Monitor side.\n1. Logs could be automatically converted to span events via `AttachLogsToActivityEvent` to get the same reliability as (events in) spans, but this changes the event message and properties too much and thus requires some more tweaking via `LogToActivityEventConversionOptions`.\n\n### Spans\n\n\u003e ✔️ Using spans for auditing purposes is supported. This includes span events via `ActivityEvent`, added to (completed) spans via `Activity.AddEvent`.\n\n**On time export of spans** to Azure Monitor is made possible by [ForceFlush](https://opentelemetry.io/docs/specs/otel/trace/sdk/#forceflush), implemented via `AutoFlushActivityProcessor`. This processor is in [OpenTelemetry.Extensions](https://www.nuget.org/packages/OpenTelemetry.Extensions).\n\n1. To confirm that it reliably exports completed spans, please see the `FailFastTest` in [IntegrationTest](/IntegrationTest/).\n1. See https://github.com/open-telemetry/opentelemetry-specification/issues/2944#issuecomment-1319536765 for more context.\n1. It doesn't check the flush status but does wait until it's done, please see: https://github.com/open-telemetry/opentelemetry-dotnet-contrib/issues/2721.\n1. [What is guaranteed on a completed Activity?](https://github.com/open-telemetry/opentelemetry-dotnet/discussions/6266)\n\n## Caveats\n\n1. Baggage is in a rough state, see:\n   - Conflicting implementations: https://github.com/open-telemetry/opentelemetry-dotnet/issues/5667\n   - Azure Function support not complete: https://github.com/Azure/azure-functions-host/issues/11026\n   - Streamlining attempt in .NET: https://github.com/dotnet/runtime/issues/112803\n   - Service Bus (messaging) support missing: https://github.com/Azure/azure-sdk/issues/6959\n1. Creating new root traces is done via some (abstracted away) tricks in this project, learn more here https://github.com/open-telemetry/opentelemetry-dotnet/issues/984\n\n## References\n\n1. [OpenTelemetry .NET Logs](https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/logs#logger-management)\n1. [OpenTelemetry .NET Contrib](https://github.com/open-telemetry/opentelemetry-dotnet-contrib)\n1. [Go OTEL Audit](https://github.com/microsoft/go-otel-audit), a package for auditing Go code for Microsoft compliance purposes\n1. [Audit.NET](https://github.com/thepirat000/Audit.NET)\n\n## License\n\nThis work is licensed under a\n[Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License][cc-by-nc-sa].\n\n[![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa]\n\n[cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/\n[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png\n[cc-by-nc-sa-shield]: https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferwinkramer%2Fotel-business","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ferwinkramer%2Fotel-business","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferwinkramer%2Fotel-business/lists"}