{"id":24307867,"url":"https://github.com/expecho/semanticlogging.eventhub","last_synced_at":"2025-09-26T13:30:45.320Z","repository":{"id":35962259,"uuid":"40252426","full_name":"Expecho/SemanticLogging.EventHub","owner":"Expecho","description":"SemanticLogging.EventHub is a collection of sinks for the Semantic Logging Application Block that exposes Event Source events to an Azure Event Hub.","archived":false,"fork":false,"pushed_at":"2018-12-07T13:43:19.000Z","size":73,"stargazers_count":6,"open_issues_count":0,"forks_count":4,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-01-06T18:34:59.822Z","etag":null,"topics":["azure","azure-event-hub","azure-event-hubs","azure-eventhub","etw","event-hubs","eventsource","logging","semantic-logging","structured-logging"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Expecho.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-05T15:12:05.000Z","updated_at":"2018-12-11T09:05:41.000Z","dependencies_parsed_at":"2022-09-14T02:53:04.117Z","dependency_job_id":null,"html_url":"https://github.com/Expecho/SemanticLogging.EventHub","commit_stats":null,"previous_names":["deheersoftware/semanticlogging.eventhub"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Expecho%2FSemanticLogging.EventHub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Expecho%2FSemanticLogging.EventHub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Expecho%2FSemanticLogging.EventHub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Expecho%2FSemanticLogging.EventHub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Expecho","download_url":"https://codeload.github.com/Expecho/SemanticLogging.EventHub/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234310895,"owners_count":18812166,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","azure-event-hub","azure-event-hubs","azure-eventhub","etw","event-hubs","eventsource","logging","semantic-logging","structured-logging"],"created_at":"2025-01-17T04:20:07.430Z","updated_at":"2025-09-26T13:30:45.022Z","avatar_url":"https://github.com/Expecho.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"## SemanticLogging.EventHub\nThe SemanticLogging.EventHub project provides sinks for the [Semantic Logging Application Block](https://msdn.microsoft.com/en-us/library/dn775014(v=pandp.20).aspx) that exposes Event Source events to an Azure Event Hub. There is an HTTPS based sink and an AMQP based sink.\n\nThese sinks are also available as a Nuget package: https://www.nuget.org/packages/SemanticLogging.EventHub/ \n\nRun the SemanticLogging.EventHub.Processor console application for an example of how to process the events.\n\n## AMQP based sink Usage\n\nMinimal setup for the AMQP based sink:\n```c#\nvar listener = new ObservableEventListener();\n\nlistener.LogToEventHubUsingAmqp(\n    \"Endpoint=sb://my-eventhub-ns.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[your key]\",\n    \"my-eventhub\"\n  );\n  \nlistener.EnableEvents(MyEventSource.Log, EventLevel.LogAlways);\n```\n\na partition key can optionally be provided using the partitionKey parameter. If no partition key is supplied the data is sent to the eventhub and distributed to various partitions in a round robin manner. \n\n## HTTPS based sink Usage\n\nMinimal setup for the HTTPS based sink:\n```c#\nvar listener = new ObservableEventListener();\n\nlistener.LogToEventHubUsingHttp(\n    \"my-eventhub-ns\",\n    \"my-eventhub\",\n    \"dev01\"\n    \"[your token]\"\n  );\n  \nlistener.EnableEvents(MyEventSource.Log, EventLevel.LogAlways);\n```\n\nYou can use this tool to generate a sas token: https://github.com/sandrinodimattia/RedDog/releases/tag/0.2.0.1\n\n## Out-of-process logging\n\nA sample configuration for out-of-process logging using a windows service (see https://msdn.microsoft.com/en-us/library/dn774996(v=pandp.20).aspx):\n\n```xml\n\u003c?xml version=\"1.0\" encoding=\"utf-8\" ?\u003e\n\u003cconfiguration xmlns=\"http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw\"\n               xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n\t\t\t   xmlns:etw=\"http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw\"\n               xsi:schemaLocation=\"http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw SemanticLogging-svc.xsd\"\u003e\n  \n  \u003csinks\u003e\n\t\u003ceventHubAmqpSink xmlns=\"urn:dhs.sinks.eventHubAmqpSink\" name=\"eventHubAmqpSink\" type =\"SemanticLogging.EventHub.EventHubAmqpSink, EnterpriseLibrary.SemanticLogging.EventHub\"\n\t\teventHubConnectionString=\"Endpoint=sb://my-eventhub-ns.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[your key]\"\n\t\teventHubName=\"my-eventhub\"\n\t\tbufferingIntervalInSeconds=\"30\"\n\t\tbufferingCount=\"100\"\n\t\tbufferingFlushAllTimeoutInSeconds=\"5\"\n\t\tmaxBufferSize=\"3000\"\u003e\n\t\t\u003cetw:sources\u003e\n\t\t\t\u003cetw:eventSource name=\"DeHeerSoftware-PlanCare2\" level=\"Warning\" /\u003e\n\t\t\u003c/etw:sources\u003e\n\t\u003c/eventHubAmqpSink\u003e\n\t\u003ceventHubHttpSink xmlns=\"urn:dhs.sinks.eventHubHttpSink\" name=\"eventHubHttpSink\" type =\"SemanticLogging.EventHub.EventHubHttpSink, EnterpriseLibrary.SemanticLogging.EventHub\"\n\t\teventHubNamespace=\"my-eventhub-ns\"\n\t\teventHubName=\"my-eventhub\"\n\t\tpartitionKey=\"dev01\"\n\t\tsasToken=\"[your token]\"\u003e\n\t\t\u003cetw:sources\u003e\n\t\t\t\u003cetw:eventSource name=\"DeHeerSoftware-PlanCare2\" level=\"Warning\" /\u003e\n\t\t\u003c/etw:sources\u003e\n\t\u003c/eventHubHttpSink\u003e\n  \u003c/sinks\u003e\n\u003c/configuration\u003e\n```\n\nYou can use this tool to generate a sas token: https://github.com/sandrinodimattia/RedDog/releases/tag/0.2.0.1\n\n## Buffering and batch size\n\nBoth sinks support buffering of events before publishing to the event hub. It is important to note that a single batch must not exceed the 256 KB limit of an event. Additionally, each message in the batch uses the same publisher identity. It is the responsibility of the sender to ensure that the batch does not exceed the maximum event size. The number of events that trigger a publish can be set using the bufferingCount variable. Set the bufferingCount to 1 to disable buffering.\n\n### Autosized buffering\n\nCurrently there is a limit on the maximum size of a message, or a batch of messages, of 256 KB for the Azure Service Bus. There is build in support for using an autosized buffer mechanism. Events are buffered until the limit is reached and then send to the Event Hub. Any leftover event will be submitted in a new batch. To enable autosized buffering set the bufferingCount to 0. Both the amqp and the https based sink support this feature.\n\n```c#\nlistener.LogToEventHubUsingAmqp(\n    \"Endpoint=sb://my-eventhub-ns.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[your key]\",\n    \"my-eventhub\",\n    bufferingCount: 0\n  );\n```\n\n## Consuming events\n\nOne possibility is to create your own EventProcessor, a simplified implementation could like like this:\n\n```c#\n    public class EventProcessor : IEventProcessor\n    {\n        public Task OpenAsync(PartitionContext context)\n        {\n            return Task.FromResult\u003cobject\u003e(null);\n        }\n\n        public async Task ProcessEventsAsync(PartitionContext context, IEnumerable\u003cEventData\u003e events)\n        {\n                if (events == null)\n                {\n                    return;\n                }\n                var eventDataList = events as IList\u003cEventData\u003e ?? events.ToList();\n\t\t\t\n\t\tforeach (var eventData in events)\n            \t{\n                \tdynamic data = JsonConvert.DeserializeObject(Encoding.Default.GetString(eventData.GetBytes()));\n                \t// do something with data\n            \t}\n\t\t\t\t\n                await context.CheckpointAsync();\n        }\n\n        public async Task CloseAsync(PartitionContext context, CloseReason reason)\n        {\n                if (reason == CloseReason.Shutdown)\n                {\n                    await context.CheckpointAsync();\n                }\n        }\n    }\n```\n\n\n## How do I contribute?\n\nPlease see [CONTRIBUTE.md](/CONTRIBUTE.md) for more details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexpecho%2Fsemanticlogging.eventhub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fexpecho%2Fsemanticlogging.eventhub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexpecho%2Fsemanticlogging.eventhub/lists"}